MCN: Modulated Convolutional Network
47
FIGURE 3.5
Accuracy with different K for 20-layer MCNs with width 16-16-32-64 on CIFAR-10.
columns show the accuracies of U-MCNs and MCNs, respectively. The performance in the
last three columns shows that the accuracy of MCNs only decreases slightly when binarized
filters are used. Note that with a fixed number of convolutional layers, the performance of
MCNs increases with larger network width. At the same time, the number of parameters
also increases. Compared to LBCNN, the parameters of the MCNs are much fewer (61 M
vs. 17.2 M), but the performance of the MCNs is much better (92.96% vs. 95.30%). Also,
the last three columns show that MCNs have achieved performance similar to U-MCNs and
WRNs.
3.4.5
Model Effect
Learning convergence: The MCNs model is based on a binarized process implemented
on the Torch platform (classification). For a 20-layer MCN with width 16-16-32-64 that is
trained after 200 epochs, the training process takes about 3 hours with two 1080ti GPUs. We
plot the training and testing accuracy of MCNs and U-MCNs in Fig. 3.10. The architecture
of U-MCNs is the same as that of MCNs. Figure 3.10 clearly shows that MCNs (the blue
curves) converge at speeds similar to those of their unbinarized counterpart (the red curves).
Runtime analysis: We performed a run-time analysis to compare MCNs and LBCNN.
The runtimes of MCNs and LBCNN for all CIFAR-10 test samples are 8.7 s and 160.6 s,
Conv
3×3, 80
R+
MP
Output
Input
image
B
N
Input
image
Copy
4
MP
CNN
MCN
MP: Max Pooling
R: ReLU
BN: BatchNormlization
D: Dropout
MCcov
4×3×3, 20
R+
MP
Conv
3×3, 160
R+
MP
Conv
3×3, 320
R+
MP
Conv
3×3, 640
R+
MP
FC
1024
D
B
N
MCcov
4×3×3, 40
R+
MP
B
N
MCcov
4×3×3, 80
R+
MP
B
N
MCcov
4×3×3, 160
R+
MP
FC
1024
D
Output
FIGURE 3.6
Network architectures of CNNs and MCNs.